1. Field of the Invention
The present invention relates in general to the field of information handling systems and more specifically, to the management of disk storage systems.
2. Description of the Related Art
As the value and use of information continues to increase, individuals and businesses seek additional ways to process and store information. One option available to users is information handling systems. An information handling system generally processes, compiles, stores, and/or communicates information or data for business, personal, or other purposes, thereby allowing users to take advantage of the value of the information. Because technology and information handling needs and requirements vary between different users or applications, information handling systems may also vary regarding what information is processed, stored or communicated, and how quickly and efficiently the information may be processed, stored, or communicated. The variations in information handling systems allow for information handling systems to be general or configured for a specific user or specific use such as financial transaction processing, airline reservation, enterprise data storage, or global communications. In addition, information handling systems may include a variety of hardware and software components that may be configured to process, store, and communicate information, and may include one or more computer systems, data storage systems, and networking systems.
The amount of data that information handling systems manage continues to grow, driving the need for scalable data storage systems capable of sustaining information integrity, reliability and availability. One approach to address these needs is the implementation of redundant array of independent disks (RAID) subsystem, which can share and/or replicate data across multiple disk drives, any of which can typically be replaced (“hot swapped”) while the system is running. In its simplest implementation, RAID combines multiple hard drives into a single logical unit, and depending on the version implemented, it can also provide increased data integrity and fault tolerance. Implementations of RAID generally involves the use of a RAID controller, which manages the disks comprising the array, and in some versions of RAID, may also perform parity calculations for error detection and correction.
Most current computer file systems are based on the concept of a “block device,” which is an abstraction of the hardware (e.g., disk drive or RAID subsystem) that stores and retrieves predetermined blocks of data. In certain versions of RAID, when a data block is written on a disk in an array, a parity block is generated within the same stripe of the array. These parity blocks are generally not read unless a data read operation results in a cyclic redundancy check (CRC) error. A CRC is a type of mathematical algorithm known as a hash function. When a data block is applied against a CRC a predetermined number of unique bits is produced, generally referred to as a checksum, that can be used to detect and correct errors within the data block. Implementation of CRCs in data storage systems is popular because they are simple to implement in binary hardware, easy to analyze mathematically, and good at detecting common errors. However, they are not as useful for distinguishing between read operation errors and disk media errors.
The original RAID specification suggested a number of RAID “levels” (e.g., 0, 1, 2, 3, 4, 5) that described the manner in which information was distributed and/or replicated across the hard disks comprising the array. Those of skill in the art will be aware that each of these commonly implemented versions of RAID provide different capabilities. For example, RAID 0 offers no redundancy or fault tolerance, but is useful for aggregating disk storage. RAID 1, generally referred to as mirroring and typically implemented with just two disk drives, offers fault tolerance by allowing the data on a failed drive to be copied to the remaining drive. RAID 2, which distributed bytes of data across disks and kept associated error correcting code (ECC) stored on additional disks, has been superseded by built-in ECC capabilities on current disk drives. RAID 3 likewise distributes bytes of data across disks, but stores parity information on a dedicated disk in the array.
RAID 4 divides information into blocks of data, which is distributed across the disk drives comprising the array. Parity information is added to each of these data blocks and is then summed into associated parity blocks. In RAID 4, the parity blocks are stored on a dedicated disk drive in the array. If one of the drives in the array fails, its contents can be recalculated and reconstructed from the parity blocks stored in the dedicated disk drive. However, the ability to restore or reconstruct a failed drive may be compromised if the disk drive dedicated to storing the parity blocks fails.
RAID 5 on the other hand, provides fault tolerance by distributing parity data across all member drives in the array. There is only one parity block per stripe, and the disk used for storing each stripe's parity block is staggered from one stripe to the next. If one of the disk drives comprising a RAID 5 subsystem fails, parity blocks from the surviving disks can be read and mathematically combined with associated data blocks distributed across the surviving disks to recover the failed drive's data. However, when a RAID 5 disk fails, the array enters a degraded state and does not regain normal operation until the failed drive is replaced and the replacement drive is rebuilt. Furthermore, if a second drive fails before the rebuilding of the replacement drive is complete there will be insufficient parity data available to reconstruct both drives, generally resulting in a failure of the RAID subsystem and a total loss of data. For this reason, many RAID 5 implementations preinstall an extra, unused disk drive that can be “hot swapped” to immediately and automatically replace a failed drive in the array. The use of such a “hot spare” disk drive can reduce the window of vulnerability during which a second drive failure could cause the RAID subsystem to fail overall.
Current approaches for rebuilding a replacement disk drive can produce undesirable results, such as one or more read errors of physical blocks on the surviving RAID drives creating logical “bad” blocks, or “punctures,” on the rebuilt drive. These “punctured” blocks on the rebuilt drive, while actually “good,” are reported as physically “bad” by subsequent diagnostic processes, and in sufficient quantity they can lead to a misdiagnosed drive failure. Furthermore, once a rebuilt disk drive's physical blocks have been marked as “punctured,” they can propagate to subsequent replacement drives when they are rebuilt. Disk punctures, if propagated in sufficient numbers, can also cause replacement drives to report as being “failed” after rebuilding is complete, possibly causing unnecessary repeat service dispatches, and/or replacement of “failed” drives that are actually good.
Current drive rebuild approaches attempt to reduce the number of punctured blocks by minimizing the possibility of a RAID controller encountering physical bad blocks during rebuilds. But these approaches do not prevent logically created “bad” blocks from being propagated onto subsequent replacement drives. Currently, the only way to remedy this propagation is to perform a low level format on the physical disk and restore data from back-up sources. What is needed is a way to determine whether a physical block on a rebuilt disk drive that is marked “bad” is a result of an actual disk media error on the source drive, or “punctured” as a result of a read error during the drive rebuild process.